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Abstract 

This report looks at the problem of determining when to award bonuses for reliability 
improvement in pyrotechnic devices based on data gathered under a lot acceptance 
sampling plan. 

1 Introduction 

As government procurement contracts are now written, contractors have no incentive to 
improve the quality of the items they provide. To improve quality, the Naval Weapons 
Support Center, Crane, IN, has decided to implement a bonus system. If, as a result of 
the sampling inspection plan currently required for lot acceptance, there is evidence that 
the lot has a reliability that exceeds the minimum requirement for lot acceptance, then the 
contractor will be eligible for a bonus. The difficulty with implementing such a system of 
bonuses is that the sampling inspection plans are designed to provide clear cut criteria for 



lot acceptance or rejections. They do not readily provide a single measure of quality or 
reliability of the item being tested. 

The first item for which a bonus system will be implemented is a pyrotechnic device. 
The sampling inspection plan for this device involves sampling items from a lot. subjecting 
them to a manufacturing test and exposing them to one of three environments. An attempt 
is then made to activate each device. Specifically, a lot of 500 to 1000 devices is accepted if 
nil of the following criteria are met: 

1. Of 20 items subjected to the manufacturing test, no more than 1 can fail to activate. 

2. Of 20 items subjected to the joint manufacturing and temperature and humidity test 
no more than 1 can fail to activate. 

5. Of 32 items subjected to the joint manufacturing and vibration test no more than 2 
can fail to activate. 

4. Of 20 items subjected to the joint manufacturing and altitude test no more than 1 
can fail to activate. 

The samples in tests 1-4 are distinct . thus a total of 02 items are tested. 

Attempts have been made to estimate the reliability R of a device (here reliability is 
defined as the probability that an item would activate after the manufacturing test and 
exposure to all three environments) based on data from the sampling inspection plan ( [4] 
and [5]) . The methods used to produce lower confidence bounds LOB for this reliability 
based on data from a lot which is just barely accepted have yeilded values that are much 
lower than observed in the field. 

The most obvious reason for this discrepancy is that requiring successful activation after 
exposure lo all environments is much more stringent than requiring a device off the shelf 



to activate in practice. An operational environment would not include all these adverse 
conditions. It is also likely that items which activate after exposure to one environment 
would he more likely to activate after exposure to other environments. Unfortunately the 
presence or absence of such dependence cannot, be inferred from the data yielded by the 
current sampling inspection plan. Thus, the models that have been used to estimate the 
reliability have assumed independence of the events that a device activates after exposure to 
the three environments given that it would have activated after the manufacturing test. If 
there is positive dependence between these events then estimation under the assumption of 
independence yields reliabilities and LCB’s that are lower than they would be if dependence 
were accounted for. It should be noted that even if exact LCB\s can be found, if the 
modeling assumptions are changed to include dependence (where these new assumptions 
are supported by empirical evidence), and if the 'true* reliability could be defined and 
estimated from the available data, the I, CD's could still be rather low for the simple reason 
that the sample sizes of the sampling inspection plan are too small to give a sharper lower 
confidence bound. 

In Section 2. we find the maximum liklihood estimator (MLE) of R under the assump- 
tion of independence of the events that a device activates after exposure to the three en- 
vironments given that it has passed the manufacturing test. In the process, we uncover a 
difficulty with estimating R under this assumption. Specifically, this procedure can yield 
lower estimates of R for samples which have fewer failures. To fix this, lower bounds for 
the MLE of R under the conditional independence assumption are derived and shown to 
be lower bounds for MLE ? s under weaker and more realistic assumptions. Approximate 
95% LCB's for R and the MLE arc provided explicitly for the 2-1 realizations of the sam- 
pling plan leading to lot acceptance. In Section 5 a direct approach to awarding bonuses 



is taken. It provides an alternative to using LCD's for* decision making. Conclusions and 
recommendations are given in Section 4. 



2 Estimating Reliability 

Tests 2. 3. and 4 in the lot acceptance sampling plan involve trying to activate the device 
after it is subjected to the manufacturing test and one of the environmental tests. To 
distinguish between the outcome of these joint tests and what would have happened had 
the device been subjected to one of the tests alone, let Ei.Eo.Ez, and £* 4 be the respective 
events that a device can activate after being subjected to just the manufacturing test, the 
temperature and humidity test, the vibration test, and the altitude test. Then. R\ = P{E\) 
and Rj = P{E\ fi E t ). i = 2.3.4 are the probabilities that a device activates after being 
subjected to tests 1, 2. 3, or 1, respectively. The reliability of a device is defined as 

R = P(E\ n E-2 n £3 n £4). 

R will be estimated based on X t . the number of devices which activate out of the ii{ 
that are subjected to test i. The sampling plan uses different devices randomly chosen from 

the lot in question. Thus, it is reasonable to assume that X\ X\ are independent and 

X t ~~ B inomial(n lt Rj). i = 1 4. 



2.1 Maximum Likelihood Estimation 



The likelihood of observing X\ = xi, 

, / \ 



, X\ = £4 is given as 



'•=n 



(=i 



R x t -( 1 - R,) n '- X \ 



11 / 



( 1 ) 



1 



where 0 < R j < 1 , 0 < R t < /?j, i = 2.3,4. For t he sampling plan described in the intro- 
duction, there are 24 realizations of X \ . . . , X* that lead to lot acceptance. The maximum 
likelihood estimates (MLE’s) Ri of R x for these 24 realizations are given in Table 1 . 

Because the same device is never actually subjected to more than one of the environmen- 
tal tests, the estimators R t of R x alone are not enough to construct an MLE for R . In the 
subsequent subsections, we describe an estimator Rci for R which is based on the assump- 
tion of conditional independence. After examining the shortcomings of Rci , we provide a 
second estimate R L q which requires no assumptions concerning dependencies between test 
outcomes, and which provides a lower bound for R . 



2.2 Conditional Independence 

Additional assumptions about the dependence between the events E[ . . . . E\ are needed 
to estimate R based on the current sampling plan data. The most direct approach for 
estimating R based on this data is to assume that, given F\. that Eo. £3. F 4 are independent 
events, (see [4] and [o] for further discussion.) The MLE’s in Table 1 are also the MLE’s for 
R x under the assumption of conditional independence. With this assumption, when R\ > 0, 
we have 

R = P(E\ n E-} n £*3 n E\) 

= P{E 2 r\E^FE A \E l )P{E^ 

= P(E 2 \E 1 )P(E,\E 1 )P(E,\E 1 )P(E 1 ) 

RoR 3R4 

~ ' 



Thus the MLE Rci of R under this assumption is given by 



Table 1: MLEs for /?i, . . . , R 4 computed for the 24 realizations of the sampling plain leading 
to lot acceptance, where ( ) represents the number of failures in tests 1, ... ,4. 



a/,*, 0 .. 


*4 


R z 


*3 


*4 


(0000) 


1.000 


1.000 


1.000 


1.000 


(0001) 


1.000 


1.000 


1.000 


0.950 


(0010) 


1.000 


1.000 


0.969 


1.000 


(0100) 


1.000 


0.950 


1.000 


1.000 


(1000) 


0.989 


0.989 


0.989 


0.989 


(0011) 


1.000 


1.000 


0.969 


0.950 


(0101) 


1.000 


0.950 


1.000 


0.950 


(0110) 


1.000 


0.960 


0.969 


1.000 


(1001) 


0.986 


0.986 


0.986 


0.950 


(1010) 


0.983 


0.983 


0.969 


0.983 


(1100) 


0.986 


0.950 


0.986 


0.986 


(0020) 


1.000 


l.COO 


0.938 


1.000 


(0111) 


1.000 


0.950 


0.969 


0.950 


(1011) 


0.973 


0.975 


0.969 


0.950 


(1101) 


0.981 


0.950 


0.981 


0.950 


(1110) 


0.973 


0.950 


0.969 


0.975 


(0021) 


1.000 


1.000 


0.938 


0.950 


(0121) 


1.000 


0.950 


0.938 


1.000 


(1020) 


0.983 


0.983 


0.938 


0.983 


(1111) 


0.962 


0.950 


0.961 


0.950 


(0121) 


1.000 


0.950 


0.938 


0.950 


(1021) 


0.973 


0.975 


0.938 


0.950 


(1120) 


0.973 


0.950 


0.938 


0.975 


(1121) 


0.950 


0.950 


0.938 


0.950 
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Rci = < 



[L ^ Ik /ii > o 

R i 

0 7?1 = 0 . 

Realizations of Rci for the 24 outcomes that lead to lot acceptance are given in Table 



From Table 2 it is clear that the assumption of conditional independence is unacceptable. 
Under conditional indepence, there are severed cases where sampling plans resulting in fewer 
failures give smaller estimated value of R than with more failures. An example of this 
is the failure configuration (1.1, 1,0) with Rci = *944 compared to the case (0. 1,1.0) 
with Rci = -920. What happens in the (0, 1. 1,0) case is that the conditional indepence 
assumption will attribute a disproportionate amount of the cause of failure to the lot’s 
inability to pass environmental tests 2 mid 3. Thus, the estimated probability of passing all 
tests is downgraded too much. This contradiction will lead us to dispose of the conditional 
independence assumption altogether. We will now make the results of this phenomenon 
formal. 

Data from the lot acceptance sampling plan can be thought of as censored. For tests 2-4. 
it is unknown whether failure is due to the manufacturing test, the particular environmental 
test, or both. This lack of information, along with the conditional independence assumption, 
causes the apparent reversals in Rci observed in Table 2. To see this, suppose that we could 
determine the cause of failure for a particular device which fails to activate. For test ?. let 



• n t — x x be the total number of failures for test 



• Xij be the number of failures resulting from only the environmental exposure (these 
would have passed the simple manufacturing test); 

• ,r r v> be the number of failures resulting from only the manufacturing test, (these would 
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Table 2: The MLE for R and an approximate 95% LCB computed for the 24 realizations 
of the sampling plan that lead to lot acceptance, where ( i,j,k,l ) represent the number of 
failures in tests 1-4 respectively 



(i, fc/ 0 


Rci 


95% LCB 


(0000) 


1.000 


1.000 


(1000) 


0.989 


.900 


(0010) 


0.969 


.906 


(1010) 


0.969 


.355 


(0001) 


.950 


.350 


(0100) 


0.950 


.850 


(1001) 


0.950 


.323 


(1100) 


0.950 


.926 


(1011) 


0.944 


.302 


(1110) 


0.944 


.300 


(1111) 


0.939 


.765 


(0020) 


0.938 


.375 


(1020) 


0.937 


.313 


(1121) 


0.937 


.751 


(0011) 


0.920 


.323 


(0110) 


0.920 


.323 


(1101) 


0.920 


.775 


(1021) 


0.913 


.765 


(1120) 


0.913 


.761 


(0101) 


0.902 


.300 


(0021) 


0.391 


.775 


(0120) 


0.891 


.738 


(0111) 


0.374 


.750 


(0121) 


0.346 


.717 
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have passed the simple environmental test); 



• x t> 3 be the number of failures resulting from both the manufacturing and the environ- 
mental tests. 

The likelihood L is then proportional to 
L oc P(Ei) Xl (l - P(Ei)) ni ~ Xl 

4 

ll p{f a n E,) 1 ' P(E\ n E]) x '- x P(E{ n E,) Xi '- , P(/r 1 c n Ef)**. 

t=2 



Let n = Conditioning oil the event E[ gives 

n P(^\EiY (i - - P(^!^)) r - 3 . 

i=2 



Therefore, the MLE's of interest are 
Ht=i r ! + - r i.i 



r(Ei) = 

P(Ei\Ei) = 



■r, 






/or /= 2.3. 1. 



With these MLE’s in hand, we can explain the behavior of the estimates in the condi- 



tional independence case. When E^Ez,E\ are independent given E\. we have 



lid = 

1=2 

E 4 i 4 

t — i x i 4~ x ?, i x-r x t 

n fJ; - r *+ r *.i ’ 



When the x^[ are not observed, the maximum likelihood approach assigns values 



Ri - R, 

A 



to j t .i where R , are found by maximizing (1). This is what leads to the seemingly non- 
sensical ordering of reliabilities seen in table 2.1. The worst case was failure configuration 
(1. 1. 2. 1), for which we can compute 



■h.i = -ri.i = 0 

ort ,.95 -.9375, 

3 i = 30( ) 

' .9375 ’ 

% 0.01333. 

Alternatively, configuration (0. 1.2. 1) results in 
1 — 05 

> 2 .! - 



.05 



1 



1 - .0375, 

./’3 i = 30( ) 

v .9375 } 



A - .95, 
>1.1 = 19 — — ) 
.Oo 



Thus, the second configuration, although being clearly superior to the first, results in assign- 
ing blame to the environments as the cause of failure. Since our definition of reliability is 
the joint probability of activating under all circumstances, the failure configurations which 
appear to point to the manufacturing test as the cause of most of the failures actually yield 
greater reliabilities. 
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Allowing the maximum likelihood procedure to assign values to the censored observa- 
tions leads to an ordering of the estimates of R that are counterintuitive. In the next 
section, we will develop a bound for R which doesn’t, use conditional independence, and 
which produces an ordering of estimates of R which is more believable. 



2.3 A Reasonable Lower Bound 

An alternative approach is to note that no matter what the censored observations are. we 
still have 

V 4 r 

P(Ei) > ■ 1; ‘ 

n 

P(E,\Ei) > —i = 2.3.4. 



Thus, under the assumption of conditional independence 



ila> (S^)n^ 



Note that the lower bound of Rci . 
. Z,«=l x >\ TT ,r ' 



Elb - ( ‘ 



: >n 



(2) 



is also equal to the MLE when there is as much dependence between E \ . . . , E± as is possible. 
This maximum dependence case arises if we assume that all of the failures observed are the 
result of BOTH the manfacturors test and and environmental test. i.e. .r , 3 = /?, — ,r,*, i = 



2.3,4. 

Rlb for the failure configurations leading to lot acceptance are shown in Table 3. There 
are two advantages to using /?£# rather than Rci to estimate R . First, estimates Riq are 
ordered in a reasonable way. failure configurations with more failures always have lower 
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Table 3: The lower bound for the MLE of R computed for the 24 realizations of the sampling 
plan leading to lot acceptance where represent the number of failures in tests 1-4 

respectively. 





Ri3 


(0000) 


1.000 


(1000) 


0.939 


(0010) 


0.953 


(1010) 


0.943 


(0001) 


0.940 


(0100) 


0.940 


(1001) 


0.933 


(1100) 


0.930 


(0020) 


0.917 


(1020) 


0.907 


(0011) 


0.900 


(0110) 


0.900 


(1011) 


0.390 


(1110) 


0.390 


(0101) 


0.333 


(1101) 


0.373 


(0120) 


0.362 


(0021) 


0.362 


(1021) 


0.352 


(1120) 


0.352 


(0111) 


0.346 


(HID 


0.336 


(0121) 


0.309 


(1121) 


0.300 



values of Rlb- Second. Rib provides a lower hound for R under much weaker assumptions 
about the dependence of the test outcomes. 



Following [4], suppose that a device is more likely to activate after exposure to an 
environment if it. has activated after exposure to a previous environment. Specifically, 
suppose 

P(Es\Ei DEo) > P(Ei \Ei) 
p(E 4 \Ei n Eo n £3) > p(e 4 \Ei). 

then 

R = P{E l )P(E-2\E l )P{E :i \F^ n E- 2 )P{E A \E l (1 £> n £3) 

> P(E l )P(E 2 \E 1 )P(E 3 \E 1 )P(E A \E 1 ). 



Thus. Rib given in (2) provides a lower hound for the MLE of R under these much more 
plausible assumptions. 

2.4 Lower Confidence Bounds 

Also given in Table 2 are approximate 95% lower confidence bounds found by bootstrap- 
ping [3]. For each of the 24 realizations under consideration. 1000 bootstrap samples were 
generated as follows. Let i — 1 4. k = 1 ..... 24 represent the value of /?,• comput- 

ed for the k ^ realization. Then, for the k th realization, the bootstrap samples consist, of 
generating 1000 independent realizations of Xij. . . . , X^. where X\^ X^. are indepen- 

dent and X u k ~ B iiiomial(n t , r u k)- From each of the 1000 bootstrap samples, the MLE of 
R is computed, and the 95% lower confidence bound is taken as the b th percentile of the 
empirical distribution of these estimates of R. 



The lower confidence bound for configuration (1, 1,2, 1) is slightly higher than the lower 
confidence bound of [4]. The.se lower confidence bounds are. however, approximations and 
are possibly biased. To see how these lower confidence bounds perform, it. is a simple 
matter to compute the probabilities that they cover the true value of R under prescribed 
dependence assumptions. Returning to the conditional independence case, we can use (1) to 

compute the probability of any sampling outcome given values of R\ /? 4 . For simplicity, 

take 7?i = ... = /? 4 . so that, under conditional independence. R = R t . For R > 0.9. the 
confidence intervals given by the lower confidence bounds in Table 2 ALWAYS cover R if 
there is at least one failure. Thus, the true level of confidence for these bounds is precisely 
the probability that at least one failure occurs during lot acceptance testing. 1 — a = 1 — R 92 . 
(« = 92 is the number of tested units.) Thus, for R — 0.99. the confidence level for the 
lower bound is approximately 60%. and decreases to 37% when R increases to 0.995. 

This strange effect is due to the design of the lot acceptance plan . The plan is designed 

to ensure with 95%, confidence that a lot is accepted if min(R\ /? 4 ) > 0.99. With higher 

reliabilities, there is a very good chance that there won’t be any failures during the test, 
and we are powerless to produce an acrumte 95% lower confidence bound for R. 

In this section, we have attempted to produce an accurate point estimate and lower 
confidence bound for /?. with the ultimate goal of establishing thresholds for assigning 
bonuses. In the next section, we reapproach the problem of assigning bonuses. 

3 Alternative Formulation 

In this section, we take a fresh approach to the problem of awarding bonuses. Our decision 
will be of the form, award a bonus if 
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R(X) > c 



where R(X) is a statistic, possibly an estimate of reliability, based on the number of 

devices that activate in tests 1-4, X = (Xi, A 4 ). As we have stipulated in the previous 

section, for 7?(X) to be reasonable, it must be nondecreasing in each of its arguements, thus 

if > y,. z=l 4 then R(x) > R( y) where x = (ji x 4 ) and y = (*/i, y±). In 

the previous section, Rib has this property. 

The ultimate goal of the bonus program is to reward manufacturers who provide goods 
which are more useful in operations than the government-specified level of acceptability. 
Thus, the bonus rule will be based on the goal of rewarding all manufacturers for which 

mi.n(R\ /V 4 ) > r with probability at least 1 — o. The decision rule which accomplishes 

this is to award bonuses if 

R(X) > c* 

where c* is the largest value of c for which 

1 - q < P(R(X) > c) (3) 

for Ri = . . . = R± — r. 

As an example, take R(X) to assign numbers 1 ,24 to the outcomes leading to lot 

acceptance as listed in Table 4. Table 4 gives the cumulative probabilities P(R(X) > c) 
when R\ = . . . = /? 4 = r for different values of r. Thus, if bonuses are to be award- 
ed to manufacturers for which rniv(R \ R.\) > 0.995 with 95% chance of identifying 

these vendors based on a lot acceptance sampling plan, then the threshold score c* = 17. 
Here, bonuses are given to manufacturers which have one or zero total failures, or have 
(0. 0. 1. 1). (0, 1, 0. 1). or (0, 1.1.0) failures. Note that in this example the values were arbi- 
trarily assigned and don't take into account the fact that test 3 uses a larger sample of 32 
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while the other tests are based on a sample size of 20. 

Depending on the environment that the device will be used in, the reliability is prob- 
ably closer to a weighted average of fib,..., R\. Because these weights differ for differ- 
ent types of pyrotechnic devices and the weights themselves must be estimated based on 
the mission the devices are used in, a simple and conservative alternative is to consider 

ftmtn = 7nm(R[ /?4 ). The minimum also has the attractive feature that it’s MLE 

can be found from the lot acceptance data without assuming anything about the depen- 
dencies between E\ £4. Table 5 gives the MLE R m in = min(R\ , R\) along with 

P(R mi n > c) when R\ = . . . = ft* = r for various values of r. 

To see that this rule acheives the desired goal, in otherwords that 

1 - a < P(R(X) > c) 

for R mtn > r where c* is chosen as in (3). and R(X) is a nondecreasing function of 

X [ X+, we need the notion of stochastic ordering. The random variable X is stochas- 

* r 

ticallv greater than the random variable Y. denoted X > Y if P(X > ,r) > P{ Y > x) for 
all . r . In the multivariate extension of this notion, the random vector X is stochastically 
greater than the random vector Y if 

E[h(X)} > E[h{ Y)] 

for all real- valued functions h which are nondecreasing in their arguements for w'hich 
the expectation exists. 

Now let X[ Y4 be independent with X t ~ Binoi7n'al(n t , R t ) and V], V4 be in- 

st 

dependent with V,” Binomial(n t , r), w'here R t > r. Then X t > Y t for i = 1 4 and by 

Theorem 4.13 of [1] 

X = (A'i Y 4 )> Y = (Y ] «). 



lb 



Table 4: Cumulative probabilities for -R(X) when R\ — = r, where ( ) 

represent the number of failures in tests 1-4 respectively. 

P{R{X) > c) 



c 




R=* .99 


R=».995 


R=>.999 


24 


(0000) 


0.401 


0.534 


0.912 


23 


(0001) 


0.432 


0.697 


0.901- 


22 


(0010) 


0.510 


0.300 


0.960 


21 


(0100) 


0.591 


0.363 


0.979 


20 


(1000) 


0.772 


0.926 


0.997 


19 


(0011) 


0.793 


0.936 


0.993 


13 


(0101) 


0.314 


0.943 


0.993 


17 


(0110) 


0.340 


0.953 


0.999 


15 


(1001) 


0.355 


0.959 


1.000 


15 


(1010) 


•0.332 


0.970 


1.000 


14 


(1100) 


0.399 


0.975 


1.000 


13 


(0020) 


0.919 


0.934 


1.000 


12 


(0111) 


0.924 


0.935 


1.000 


11 


(1011) 


0.929 


0.936 


1.000 


10 


(1101) 


0.932 


0.937 


1.000 


9 


(1110) 


0.933 


0.933 


1.000 


3 


(0021) 


0.942 


0.939 


1.000 


7 


(0121) 


0.942 


■ 0.983 


1.000 


6 


(1020) 


0.947 


0.939 


1.000 


5 


(1111) 


0.943 


0.939 


1.000 


4 


(0121) 


0.943 


0.939 


1.000 


3 


(1021) 


0.950 


0.990 


1.000 


2 


(1120) 


0.950 


0.990 


1.000 


1 


(1121) 


0.950 


0.990 


1.000 
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Table 5: Cumulative probabilities for R mm when R\ = r, ...ftj = r, where ( i,j,k,l ) 
represent the number of failures in tests 1-4 respectively. 



c- 


(ij. k, i) 


R=.99 


P(Amm > C ) 
R-.995 


R®». 999 


1.000 


(0000) 


0.401 


0.634 


0.913 


a. 939 


(1000) 


0.432 


0.697 


0.931 


0.969 


(0010) 


0.636 


0.309 


0.961 


0.960 


(1010) 

(0110) 


0.667 


0.319 


0.962 


0.950 


(0001) 


0.913 


0.930 


1.000 


0.933 


(0100) 

(0011) 

(0101) 

(1001) 

(1100) 

(0111) 

(1011) 

(1101) 

(1110) 

(0020) 


0.952 


0.990 


1.000 




(0021) 

(0121) 

(1020) 

(HID 

(0121) 

(1021) 

(1121) 

(1121) 


0.950 


0.990 


1.000 
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This gives us R(X) > R( Y) so that 
P(R(X) > c *) > P(R( Y) > c') > 1 - a. 

More formally, the procedure to award bonuses tests the null hypothesis H 0 : R± > 
r, . . . , /?4 > r. Defining the nnll hypothesis in this way a.lows us to control the level of 
significance a , i.e. the probability of not awarding a bonus when one is deserved. The 
likelihood ratio test (e.g. [2]) for this problem is to award a bonus (i.e. fail to reject II 0 ) if 
R(X) > c* where R(X) is given by 

- y>7 nip, + (m - X t )ln(^-)}I(R, < r) 
n, 1 - n, 

where 1(A) is the indicator function of the set A. 

4 Conclusion 

There are several factors that make using acceptance test data to award bonuses challenging. 
They all stem from the fact that the data is gathered from a lot acceptance sampling plan 
which was not designed to estimate an overall reliability, nor was it designed to give good 
estimates of reliabilities when they are higher than the minimal lot acceptance standards. 

To use the data from the lot acceptance sampling plan, the most reasonable approach 
seems to use and its distribution for R\ = . . . = R± to decide whether to award a 

bonus. 

There are several ways that this decision rule can be improved. These initiatives were 
mentioned earlier, and include 

1. Take larger samples or sample sequentially. This will reduce the chances of awarding 
a bonus to underserving manufacturers. 



It) 



2. Construct a better measure of overall reliability. A viable option is a weighted average 

R = T, u ‘ R ' 

1=1 

where the weights ?r, represent the proportion of the devices in the field that, are 
subjected to environment L This would involve either constructing or estimating 
different weights for different types of devices. 

3. Use R = P(E i D El D £3 H £ 4 ) as the measure of overall reliability, but collect data to 
estimate the interaction between the environmental tests. R could then be estimated 
using a loglinear model based on censored observations. 
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